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(54) Pointing position detection device, presentation system and method 



(57) The present invention relates to a pointing po- 
sition detection device that can automatically sense a 
pointing position within a display region. This pointing 
position detection device comprises a CCD camera ac- 
quiring an image of a display region in which an image 



has been displayed; a binary-data processing section 
for detecting a pointing position of a shadow region or 
an actual image region that is an image region of a point- 
ing image included within an image-acquisition area, 
based on an image-acquisition signal of the CCD cam- 
era 14; and a pointing coordinates detection section. 
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BACKGROUND OF THE INVENTION 
Field of the Invention 

[0001] The present invention relates to the present in- 
vention pointing position detection device, presentation 
system, method, and an information storage medium. 

Background Art 

[0002] In case a presenter is assumed to be giving a 
presentation while viewing a screen-image that appears 
on a screen, the presenter goes through the presenta- 
tion while using a pointing device to indicate any of var- 
ious points on the screen. 

[0003] With such a system, it is important that the pre- 
senter can indicate any point on the screen in a simple 
manner, and it is also important that the system can re- 
liably perceive that pointing position and can copy the 
pointing position for image processing. 
[0004] However, when the presenter indicates de- 
sired points in a prior-art system, a problem arises in 
that a special pointing device has to be used therefor, 
which is extremely inconvenient for the user. 
[0005] If, for example, a laser pointer is used to indi- 
cate a desired point on the screen, the pointing position 
is difficult to see if the surroundings of the screen are 
not dark enough. Since the pointing position is displayed 
as a small dot, this also causes a problem in that the 
pointing position is difficult to see. 
[0006] If a pointing device such as a mouse is used 
for giving a presentation while moving a cursor position 
that is displayed on the screen, the presenter must in- 
terrupt the presentation to a certain degree, to operate 
the mouse. This causes problems in that the presenta- 
tion cannot proceed smoothly and the operation is ex- 
tremely inconvenient. 

SUMMARY OF THE INVENTION 

[0007] An objective of the present invention is to pro- 
vide a pointing position detection device, presentation 
system, and method, and a computer-readable medium 
in which a pointing position within a display region can 
be sensed automatically and accurately, without using 
a special device as a point indication tool. 
[0008] In order to achieve the objective described 
above, a pointing position detection device of the 
present invention comprises: 

image-acquisition means for acquiring an image of 
a display region in which an image is displayed; and 
position detection means for detecting a pointing 
position from an image region of a pointing image 
included within the display region, based on an im- 
age-acquisition signal of the image-acquisition 
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[0009] A method in accordance with the present in- 
vention comprises: 

an image-acquisition step of acquiring a picture of 
a display region in which an image is displayed; and 
a position detection step of detecting a pointing po- 
sition from an image region of a pointing image in- 
cluded with in the display region. 



[0010] When the presenter indicates a desired posi- 
tion in the display region in which a image is displayed 
with a predetermined indication tool such as a rod or 
15 finger, this aspect of the invention ensures that this in- 
dicated position can be detected as the pointing posi- 
tion, based on the image region of the pointing image 
generated by this indicator 

[001 1] This makes it possible to implement a pointing 
20 position detection device that is extremely convenient 
to use. 

[001 2] A pointing position detection device in another 
aspect of the present invention comprises: 

2 $ image-acquisition means for acquiring an image of 
a display region in which an image is displayed; 
reference-value setting means for setting a refer- 
ence value for separating an image region of a 
pointing image included within the display region, 

30 from an image-acquisition signal of the image-ac- 
quisition means; and 

position detection means for detecting a pointing 
posit ion from the image region of the pointing im- 
age, by comparing the image-acquisition signal and 
35 the reference value and thus extracting the image 
region of the pointing image, 
wherein the reference-value setting means is con- 
figured to be capable of modifying the reference val- 
ue. 

40 

[001 3] A pointing position detection method in this as- 
pect of the present invention comprises: 



45 



SO 



55 



an image-acquisition step of acquiring an image of 
a display region in which an image is displayed; and 
a position detection step of comparing an image- 
acquisition signal with a reference value for sepa- 
rating an image region of a pointing image and thus 
extracting the image region of the pointing image, 
then detecting a pointing position from the image 
region of the pointing image, 
wherein the position detection step comprises a 
step of modifying the reference value for separating 
the image region of the pointing image included 
within the display region from the image-acquisition 
signal of an image-acquisition means. 

[0014] This aspect of the present invention extracts 
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the image region of a pointing image created by the in- 
dicating action of an operator within a display region in 
which an image is displayed, from an image acquired 
by the image-acquisition means, and thus automatically 
detects a pointing position. 

[0015] When the image-acquisition signal that is out- 
put from the image-acquisition means is that of a mon- 
ochrome image, for example, a binary threshold value 
for extracting the image region of the pointing image 
from this image-acquisition signal could be set as the 
reference value. 

[001 6] This reference value is basically set to a value 
that is lower than the black-level brightness of an image 
acquired by the image-acquisition means. This is be- 
cause the color in the screen image that has the lowest 
brightness is black, so the brightness of the image re- 
gion of the pointing image is lower than the black bright- 
ness level. 

[001 7] If the margin between the black level of the im- 
age displayed on the display region and the reference 
value for the binary-encoding is too large in this case, a 
problem will arise in that the brightness level of the im- 
age region of the pointing image will not fall as far as 
below that reference level and it will not be possible to 
detect the image region of the pointing image. 
[001 8] Conversely, when the margin for the reference 
value is too smalj, the image target that is not the image 
region of the pointing image will be recognized errone- 
ously as the image region of the pointing image. When 
the brightness within the display region varies some- 
what due to ambient environmental conditions, for ex- 
ample, this will cause a problem in that the variation will 
be perceived erroneously as the image region of the 
pointing image. 

[0019] Therefore, if this reference value is not set to 
a value that is suitable for the environment in which the 
system is used, a problem will arise in that it will be dif- 
ficult to delect the image region of the pointing image 
appropriately. 

[0020] Another aspect of the present invention is con- 
figured to make it possible to modify the reference value 
that is used for separating the image region of a pointing 
image from an image-acquisition signal. This solves the 
problem described above and makes it possible to ac- 
curately extract the image region of a pointing image 
that is formed on the display region, and thus detect the 
pointing position automatically. 
[0021] The position detection means or step in this as- 
pect may be configured in such a manner that a prede- 
termined position of a shadow region of the pointing im- 
age included within the display region is detected as the 
pointing position. 

[0022] Alternatively, the position detection means or 
step may be configured in such a manner that a prede- 
termined position of a real-image region of the pointing 
image included within the display region is detected as 
the pointing position. 

[0023] Alternatively, the position detection means or 



step may be configured in such a manner that the point- 
ing position is detected from the image region of the 
pointing image having a rod-like shape included in the 
display region. 

5 [0024] The position detection means or step may also 
be configured in such a manner to capture a rod-shaped 
image region as the image region of the pointing image, 
based on the continuous shape of the image region in- 
cluded within the display region, and detect a tip position 

10 of the image region as the pointing position. 

[0025] A presentation system of the present invention 
comprises: 

image display means for displaying an image; 

* 5 image-acquisition means for acquiring an image of 
a display region in which an image is displayed; 
position detection means for detecting a pointing 
position from an image region of a pointing image 
included within the display region based on an im- 

20 age-acquisition signal of the imagenacquisition 
means; and 

cursor control means for controlling a position of a 
cursor included within the image displayed by the 
image display means, based on the detected point- 
2S ing position. 

[0026] A computer useable information embodied on 
computer-readable medium in this aspect of the present 
invention comprises: 

30 

information for detecting a pointing position of an 
image region of a pointing image included in a dis- 
play region, based on an image-acquisition signal 
of the display region; and 
35 information for controlling a position of a cursor in- 
cluded within an image displayed by an image dis- 
play means, based on the detected pointing posi- 
tion. 

40 [0027] A method in another aspect of the present in- 
vention comprises: 

an image display step of displaying an image; 
an image-acquisition step of acquiring an image of 

45 a display region in which an image is displayed; 

a position detection step of detecting a pointing po- 
sition of an image region of a pointing image includ- 
ed within the display region from an image^acquisi- 
tion signal of the image-acquisition means; and 

so a cursor control step of controlling a position of a 
cursor included within the image displayed by the 
image display means, based on the detected point- 
ing position. 

55 [0028] With this aspect of the invention, a presenter 
can point at any desired position on the display region 
with a predetermined indication tool, such as a rod- 
shaped indication tool or a finger, and that pointing po- 
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sition will be detected and the position of a cursor will 
be controlled. This ensures that the presentation can 
proceed smoothly. 

[0029] Note that the indication tool that creates the 
shadow or actual image of the pointing image is prefer- 
ably formed to have a slightly larger pointing position at 
the tip of a rod-shaped indication tool, and a non-reflec- 
tive material is used to make this pointing position. This 
ensures that the tip portion of the indicator rod within the 
acquired image can form a shadow or actual image re- 
liably, and thus the pointing position can be detected 
more accurately. 

[0030] The presentation system in accordance with 
this aspect of the invention controls the position of a cur- 
sor displayed on the display region, based on the de- 
tected pointing position. This ensures that the cursor 
moves to follow the position indicated by the presenter 
so that the pointing position can be displayed in an ex- 
tremely easy-to-see manner, not only for the presenter 
but also for the audience watching the presentation. As 
a result, the presentation can proceed in a form that is 
natural to both the presenter and the audience, making 
it possible to implement a presentation system that is 
convenient to use and also easy to understand. 
[0031] The above described position detection 
means, information, or step could be configured in such 
a manner that a predetermined position of a shadow re- 
gion of the pointing image included within the display 
region is detected as a pointing position. 
[0032] This makes it possible for any pointing position 
indicated by the presenter within the display region to 
be detected as a shadow region on the display region. 
[0033] Alternatively, this position detection means, in- 
formation, or step could be configured in such a manner 
that a predetermined position of a real-image region of 
the pointing image included within the display region is 
detected as a pointing position. 
[0034] This makes it possible for any pointing position 
indicated by the presenter within the display region to 
be detected as a real-image region on the display re- 
gion. 

[0035] Thus, when the presenter indicates a certain 
position with an indication tool such as a rod, this aspect 
of the invention makes it possible for that pointing posi- 
tion to be detected as a shadow region or real-image 
region of the indication tool on the display region. 
[0036] This position detection means, information, or 
step is preferably configured in such a manner that a 
rod-shaped image region included within the display re- 
gion is captured as the image region of the pointing im- 
age, and a tip position of the image region is detected 
as the pointing position. 

[0037] This ensures that detection targets other than 
the rod-shaped image region are determined to be noise 
within all the possible detection targets and thus are not 
perceived to be the pointing image. This enables accu- 
rate pointing position detection that is not affected by 
factors such as noise and environment conditions. 



[0038] In addition, the tip of the image region of the 
rod-shaped pointing image is detected as the pointing 
position, so that the presentation can proceed with a 
sensation as if the user is giving the presentation while 
5 using a finger or rod to indicate as if with a black plate 
or the like. 

[0039] This position detection means, information, or 
step is also preferably configured in such a manner that 
a rod-shaped image region is captured as the image re- 
io gion of the pointing image, based on the continuous 
shape of the image region included within the display 
region. 

[0040] This configuration ensures that the pointing im- 
age can be reliably separated from noise images com- 

is prised within the acquired image, making it possible to 
detect the pointing position even more reliably. 
[0041 ] The cu rsor control means, information, or step 
in this aspect is preferably configured in such a manner 
that a plurality of cursors is displayed simultaneously 

20 within the display region and also each cursor position 
is controlled independently. 

[0042] Another aspect of the cursor control means, in- 
formation, or step comprises means, information, or a 
step for fixedly displaying the cursor at a desired position 

& within the display region. 

[0043] This configuration ensures that a cursor can be 
made to move to folbw the position indicated by the pre- 
senter, so that the audience can perceive the position 
indicated by the presenter as the position of the cursor, 

30 in an easily understandable fashion. 

[0044] In a still further aspect of the present invention, 
cursor control is provided for a plurality of pointing im- 
ages, as described in the example below. Assume, for 
example, that the pointing position is to be displayed in- 

-35 dividually for each of the presenter and a participant 
When a cursor at the position of the pointing image in- 
dicated by the participant (assistant) can be controlled 
in addition to control over the cursor position indicated 
by the presenter, they can both see at a glance a location 

40 that requires explanation or a location of a question. This 
makes it possible to implement natural discussions and 
conferences, enabling highly understandable presenta- 
tions. 

[0045] Similarly, when the presenter wishes to com- 
45 pare two locations on the displayed screen-image, the 
presenter can set a first cursor at the positfon of a first 
explanatory point by pointing at it directly, then set the 
cursor to be fixed at that position and point directly at 
the location of the comparison point. This enables the 
so audience to see the two marker locations simultaneous- 
ly, so that the audience can simply search for the two 
marker locations. 

[0046] It is also possible to hold the cursor position 
then return to the presentation smoothly when the pre- 
55 senter is searching through reference material and thus 
is separated from the display region and cannot indicate 
it. This makes it possible to efficiently implement a pres- 
entation, without causing the attention of the partici- 
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pants to wander 

[0047] A further aspect of this cursor control means, 
information, or step preferably controls a cursor position 
to indicate the detected pointing position, at a position 
having no overlap with the image region of the pointing s 
image. 

[0048] Thus the position at which the cursor is dis- 
played is located such that it does not overlap the posi- 
tion indicated by the presenter on the display region, so 
that the pointing position can be conveyed to the audi- 10 
ence watching the presentation in an easy-to-under- 
stand and accurate fashion. 

[0049] The above described cursor control means, in- 
formation, or step may control a cursor size to be dis- 
played according to a tip size of an image region of the is 
pointing image having a rod like shape. 
[0050] For example, when the tip of the pointing im- 
age covering region is small, the pointing position or the 
cursor can be made easier for the audience to see, by 
displaying the cursor larger. 20 
[0051] A device or system in this aspect of the present 
invention may comprise: 

a screen having the display region, 
wherein the image display means comprises an im- 25 
age projection means for projecting an image onto 
the display region; and 

optical means of the image-acquisition means con- 
trols image-acquisition conditions in linkage with 
optical means of the image projection means. 30 

[0052] In a computer useable information embodied 
on computer-readable medium as another aspect of the 
present invention, information for controlling optical 
means of the image-acquisition means preferably com- 35 
prises information for controlling image-acquisition con- 
ditions in linkage with optical means forming an image 
projection means for projecting an image onto the dis- 
play region. 

[0053] Ordinarily, the presenter adjusts the focus and 40 
zoom of the optical means, such as a lens, of the image 
projection means while watching the screen-image dis- 
played on the display region. On the other hand, optical 
adjustment of the image-acquisition means is often 
done while the presenter is watching the acquired im- 45 
age. With this aspect of the invention, the presenter is 
released from making these two troublesome adjust- 
ment operations by control that links the optical adjust- 
ment of the image-acquisition means to the optical ad- 
justment of the image projection means, thus making 50 
the system more convenient. 
[0054] A device or presentation system in this aspect 
of the present invention preferably comprises: 

a display having the display region, 55 
wherein the image projection means is disposed in 
such a manner as to project an image from in front 
of or behind the screen. 



[0055] A front-projection type of presentation system, 
which projects an image from the front of a screen that 
is a display region, makes it possible for the presentation 
to proceed while a large screen image for the presenta- 
tion is projected onto the screen from the image display 
means. However, in this case, it is often difficult to touch 
the screen image directly, or point to a position thereon. 
In this case too, this aspect of the invention makes it 
possible for the presenter, who is positioned between 
the image-acquisition means and the screen, to cause 
a desired position to be perceived as a pointing position, 
by creating a shadow on the screen in such a manner 
as to interrupt part of the light path of the projection 
means, or by creatnng an actual image of the indication 
means within the acquired image in such a manner as 
to interrupt part of the light path of the image-acquisition 
means. 

[0056] Similarly, in a rear-projection type of presenta- 
tion system, which projects an image from the rear of 
the display region, the presenter can cause a shadow 
region or real-image region of a pointing image to ap- 
pear at a position on the display region indicated by the 
presenter. Therefore the pointing position can be detect- 
ed from the shadow or actual image thereof. 
[0057] A still further aspect of the presentation system 
in accordance with the present invention comprises: 

image display means for displaying an image; 
image-acquisition means for acquiring an image of 
a display region in which an image is displayed; 
reference-value setting means for setting a refer- 
ence value for separating an image region of a 
pointing image included within the display region 
from an image-acquisition signal of the image-ac- 
quisition means; 

position detection means for detecting a pointing 
position from the image region of the pointing im- 
age, by comparing the image-acquisition signal and 
the reference value and thus extracting the image 
region of the pointing image; and 
cursor control means for controlling a position of a 
cursor included within the image displayed by the 
image display means, based on the detected point- 
ing position, 

wherein the reference-value setting means is con- 
figured to be capable of modifying the reference val- 
ue. 

[0058] A computer useable information embodied on 
computer-readable medium for controlling a presenta- 
tion system, as a still further aspect of the present in- 
vention, comprising: 

reference-value setting information for setting a ref- 
erence value for separating an image region of a 
pointing image included within a display region from 
an image-acquisition signal of image-acquisition 
means; 
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position detection information for detecting a point- 
ing position from the image region of the pointing 
image, by comparing the image-acquisition signal 
and the reference value and thus extracting the im- 
age region of the pointing image; and s 
cursor control information for controlling a position 
of a cursor included within an image displayed by 
image display means, based on the detected point- 
ing position, 

wherein the reference-value setting information io 
comprises information for modifying the reference 
value. 

[0059] As a still further aspect of the present inven- 
tion, a method of detecting a pointing position for a pres- is 
entatbn comprises: 

an imagenacquisition step of acquiring an image of 
a display region in which an image is displayed; and 
a position detection step of comparing an image- 20 
acquisition signal with a reference value for sepa- 
rating an image region of a pointing image and thus 
extracting the image region of the pointing image, 
then detecting a pointing position from the image 
region of the pointing Image; and 25 
a cursor control step of controlling a position of a 
cursor included within an image displayed by an im- 
age display means, based on the detected pointing 
position, 

wherein the position detection step comprises a 30 
step of modifying the reference value for separating 
the image region of the pointing image included 
within the display region from the image-acquisition 
signal of an image-acquisition means. 

[0060] This aspect of the invention makes it possible 
to modify the reference value that is used for separating 
the image region of a pointing image from an image- 
acquisition signal. This ensures that the image region of 
a pointing image that is formed on the display region can 40 
be extracted accurately and, as a result, the pointing po- 
sition can be detected automatically and the position of 
the cursor can be controlled easily, enabling the pres- 
entation to proceed smoothly. 

[0061] The position detection means, information, or *s 
step may be configured in such a manner that a prede- 
termined position of a shadow region of the pointing im- 
age included within the display region is detected as the 
pointing position. 

[0062] Alternatively, this position detection means, in- so 
formation, or step may be configured in such a manner 
that a predetermined position of a real-image region of 
the pointing image included within the display region is 
detected as the pointing position. 

[0063] The position detection means, information, or ss 
step is preferably configured in such a manner that a 
rod-shaped image region included within the display re- 
gion is captured as the image region of the pointing im- 



age, and a tip position of the pointing image is detected 
as the pointing position. 

[0064] The position detection means, information, or 
step also preferably captures a rod-shaped image re- 
gion as the image region of the pointing image, based 
on the continuous shape of the image region included 
within the display region. 

[0065] The device and system in this aspect of the 
present invention preferably further comprises: 

calibration pattern image generation means for dis- 
playing a predetermined calibration pattern image 
from the image display means to the display region, 
wherein the reference-value setting means auto- 
matically sets a reference value for extracting the 
image region of the pointing image, based on data 
of the calibration pattern image and the image-ac- 
quisition signal. 

[0066] A computer useable information embodied on 
computer-readable medium in this aspect of the present 
invention preferably comprises calibration pattern con- 
trol information for displaying a predetermined calibra- 
tion pattern image from the image display means to the 
display region, 

wherein the reference-value setting information 
comprises information for automatically setting a refer- 
ence value for extracting an image region of the pointing 
image, based on data of the calibration pattern image 
and the image-acquisition signal. 
[0067] More specifically, this calibration pattern image 
is projected from the image display means towards the 
display region when the image display means starts op- 
erating, or during a mid-operation update. 
[0068] In this case this time, the reference-value set- 
ting means automatically sets the reference value that 
is used for extracting the image region of the pointing 
image from the acquired image, based on an image-ac- 
quisition signal obtained from the acquisition of an im- 
age of the displayed calibration pattern and brightness 
data for this calibration pattern image. If the image-ac- 
quisition signal is a monochrome image signal, for ex- 
ample, the reference value is set automatically to have 
an optimaJ margin with respect to the brightness level of 
black. 

[0069] In this case, the device and system in this as- 
pect preferably further comprises: 

indication means for setting a reference value, 

wherein, when a reference-value setting instruction 

is input by the indication means: 

the calibration pattern image generation means 

causes the display of a calibration pattern image 

from the image display means on the display region; 

and 

the reference-value setting means automatically 
sets a reference value for extracting the image re- 
gion of the pointing image, based on data of the cal- 
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ibration pattern image and the image-acquisition 
signal. 

[0070] Even with a presentation system that is used 
in the same location, the optimal reference value often 
varies according to factors such as the time at which it 
is used and the ambient environment. For example, the 
optimal reference value will differ when the system is 
used during the day from when it is used at night. Even 
when the system is used at the same time of day, this 
optimal reference value will change depending on 
whether the day is sunny or cloudy. , 
[0071] In addition, if the presentation extends over a 
long period of time, it can often happen that the ambient 
brightness will change between the start of the presen- 
tation and a point at which a certain amount of time has 
elapsed, so that the optimal reference value will also 
change. 

[0072] I n such a case too, this aspect of the invention 
could be configured in such a manner that a calibration 
pattern image is displayed onto the display region from 
the image display means, by using the indication means 
to specify that the reference value is reset. This ensures 
that the reference-value setting means automatically re- 
sets the optimal reference value on the basis of the im- 
age-acquisition signal for this calibration pattern and 
calibration pattern data. 

[0073] The reference-value setting means preferably 
divides the display region into a plurality of regions and 
sets an individual reference value for each of the divided 
regions. 

[0074] In such a case, the calibration pattern image 
generation means is preferably configured in such a 
fashion that a predetermined calibration pattern image 
having a calibration pattern for each of these partial ar- 
eas is displayed on the display region. 
[0075] In another aspect of the invention, this display 
region is preferably divided into at least a comer area 
that can easily become comparatively dark and a center 
area that can easily become relatively bright, and an in- 
dividual reference value is set for each of these partial 
areas. 

[0076] In a stiil further aspect of the computer useable 
information embodied on computer-readable medium of 
the present invention, the reference-value setting infor- 
mation preferably comprises information for dividing the 
display region into a plurality of regions and automati- 
cally setting an individual reference value for each of the 
divided regions. 

[0077] When a projector is used as the image display 
means and the display region is curved or the brightness 
of the black level of the screen image becomes uneven 
due to variations in the brightness of the projected im- 
age, it will be necessary set an optimal reference value 
to suit each area. 

[0078] With this aspect of the invention, the image re- 
gion of the pointing image can be extracted reliably from 
the acquired image and the pointing position can be de- 



tected, even if there are variations in brightness over the 
display region, by dividing the display region into a plu- 
rality of regions and setting an individual reference value 
for each partial area. 
s [0079] The calibration pattern image generation 
means preferably displays a predetermined calibration 
pattern image formed by combining of a plurality of re- 
gions having different brightnesses on the display re- 
gion. 

w [0080] In other words, since it often happens that the 
calibration pattern displayed from the image display 
means during the reference value setting process sets 
black, with the lowest brightness level, as the reference, 
for example, the entire area of the calibration pattern 

« image could be set to a completely black level. However, 
when the same black is displayed as an image on the 
display region, the brightness level obtained by the im- 
age-acquisition means when it acquires that black will 
be slightly different if the surroundings are completely 

20 black and if there are areas other than black in the sur- 
roundings. 

[0081] With this aspect, the configuration is such that 
a calibration pattern image formed of a combination of 
a plurality of regions having different brightnesses is dis- 
ss played and the reference value can be set from this cal- 
ibration pattern image, in such a manner that the refer- 
ence value can be set reliably and also to the optimum 
value even in the above cases. This makes it possible 
to extract the image region of the pointing image more 
30 accurately from the image-acquisition signal, and thus 
detect the pointing position. 

[0082] In particular, a suitable value can be set for the 
reference value for each of the divided region, by a con- 
figuration in which a combined pattern of a plurality of 
35 regions having different brightnesses is displayed in the 
display region, as a calibration pattern image set for 
each divided region. 

[0083] Yet another aspect of the presentation system 
of the present invention comprises: 

40 

image display means for displaying an image; 
image-acquisition means for acquiring an image of 
the display region; and 

processing means for detecting a pointing position, 
^5 based on an image-acquisition signal of the image- 
acquisition means. 

[0084] A method of detecting a pointing position for a 
presentation in a further aspect of the present invention 
50 comprises: 

an image-acquisition step of acquiring an image of 
a display region in which an image is displayed; and 
a step of detecting a pointing position from an image 
55 region of a pointing image, based on an image-ac- 
quisition signal. 

[0085] In this case, the processing means or position 
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detection step may be configured in such a manner that 
the pointing position is detected from a shadow region 
of the pointing image of the presenter included within 
the display region. 

[0086] Alternatively, this processing means or posi- s 
tion detection step may be configured in such a manner 
that the pointing position is detected from a real-image 
region of the pointing image of the presenter included 
within the display region. 

[0087] When the presenter indicates a certain posi- 10 
tion with a finger or a rod-like indication tool in this as-' 
pect of the invention, that pointing position is detected 
as a region of the shadow or the actual image of the 
finger or indication tool on the display region, and the 
pointing position is copied for data processing. is 
[0088] This makes it possible to implement a system 
that enables a presentation which feels more human, in 
a form that is particularly natural to both the presenter 
and the audience. 

[0089] The processing means or position detection 20 
step in this aspect preferably comprises a pointing po- 
sition detection means or step for detecting the pointing 
position of the presenter from the region of the rod- 
shaped pointing image included within the display re- 
gion. 2S 
[0090] The processing means or position detection 
step is preferably further configured to enable the de- 
tection of a plurality of the pointing positions included 
within the display region. 

[0091] In an even further aspect of the presentation 30 
system in the present invention, it is preferable that the 
image display means comprises an image projection 
means for projecting an image on a screen, and is dis- 
posed in such a manner as to project an image from in 
front of or behind the screen. 35 
[0092] Optical means of the image-acquisition means 
is preferably configured to enable control of image-ac- 
quisition conditions, linked to optical means of the image 
projection means. 

[0093] The image-acquisition means is preferably 40 
configured in such a manner that the image-acquisition 
area is freely adjustable with respect to the display re- 
gion. 

[0094] This makes it possible to implement a system 
that is extremely convenient to use, because the image- 45 
acquisition area can be set as required to suit the envi- 
ronment in which it is used. 

[0095] In this case, it is more preferable to configure 
the image-acquisition means and the image display 
means as an integral unit, to increase the convenience so 
of the system. 

[0096] This image-acquisition means is preferably 
configured in such a fashion that the image-acquisition 
direction thereof can be adjusted vertically. 
[0097] This configuration makes it possible to prevent 55 
any distortion of the acquired image caused by the en- 
vironment in which the. image display means and im- 
age-acquisition means are used, thus making it possible 



to detect the pointing position more accurately from the 
acquired image. 

[0098] The processing means or position detection 
step preferably comprises: 

cursor control means or a cursor control step, for 
controlling a cursor position included within the image 
displayed by the image display means, based on the de- 
tected pointing position. 

[0099] The cursor control means or step is further 
preferable to display a plurality of cursors simultaneous- 
ly within the display region and control each cursor po- 
sition independently. 

[0100] This cursor control means or step is further 
preferable to comprise a position-fixing means or step 
for fixedly displaying the cursor at a desired position 
within the display region. 

[0101] The system preferably comprises a polarizing 
plate which is provided in a light path of the imagenac- 
quisition means and absorbs light projected from the im- 
age projection means. 

[0102] In this case, another polarized plate is prefer- 
ably provided in the light path of the image projection 
means as well, where this polarized plate has a direction 
of polarization that differs from that of the polarized plate 
provided in the light path of the image-acquisition 
means. 

[0103] This configuration ensures that the pointing 
position can be detected with comparatively little ad- 
verse effect from the light projected from the image pro- 
jection means. 

[0104] The presentation system in this aspect of the 
present invention preferably comprises illumination 
means for lighting a low-brightness level region of the 
display region. 

[0105] In this case, the illumination means is prefera- 
bly configured so that the direction of lighting thereof is 
freely adjustable with respect to the display region, in 
such a manner that any area having variations in the 
brightness of the display image caused by factors such 
as arabient lighting or environmental conditions, such 
as large corner areas that are affected by shading, can 
be selectively lit. 

[01 06] This ensures that the effects of shading on the 
screen can be reduced, making it possible to detect the 
pointing position more accurately. 
[0107] The presentation system in this aspect of the 
present invention could further comprise: 

function input means to which a predetermined 
function is allocated; and 

means for performing data processing according to 
a function being input. 

[0108] This ensures that the presentation can pro- 
ceed while the presenter uses a mouse to perform an 
operation that is similar to a screen-image operation, by 
allocating a function to the left-click or right-click opera- 
tion of a mouse. For example, a function such as drag- 
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ging, underlining, or handwriting input could be allocat- 
ed, so that the presentation can be given white using 
that function. 

[0109] In addition, it is possible to use the cursor con- 
trol means to indicate a plurality of points or enable a s 
detailed discussion in a simple manner, by allocating an 
instruction for fixing the display of the position at which 
the cursor is displayed on the display region and another 
instruction for returning from the fixed display, thus in- 
creasing the effectiveness of the presentation. 10 
[0110] In this case, when the presenter uses a rod- 
shaped indication tool for pointing, the function input 
means could be configured in such a manner that it is 
formed either integrally with that indication tool or sep- 
arately from that indication tool, so that the presenter is 
can operate it while holding it in a hand or wearing it on 
the body. 

BRIEF DESCRIPTION OF THE DRAWINGS 

20 

[0111] 

Fig. 1 is a schematic view illustrating a front type of 
presentation system of the present invention; 
Fig. 2A is a schematic view illustrating a rear type 25 
of presentation system incorporating a CCD cam- 
era of the present invention, and Fig. 2B is an illus- 
trative view of a rear type of presentation system 
with an attached CCD camera of the present inven- 
tion; 30 
Fig. 3 is an illustrative view of an example of an in- 
dicator rod used in the present embodiment; 
Pig. 4A is an illustrative view of a presentation and 
Fig. 4B is an illustrative view of the shadows of the 
presenter and the indicator rod and the actual im- 35 
age of the indicator rod within an image that is ac- 
quired during this presentation; 
Fig. 5 is a functional block diagram of the system of 
the present embodiment; 

Fig. 6 is a detailed functional block diagram of the 40 
system of the present embodiment; 
Fig. 7A is an illustrative view of the steps of extract- 
ing a shadow of a pointer from the image-acquisi- 
tion signal of the CCD camera and detecting a point- 
ing position, and Fig. 7B is an illustrative view of the 45 
image processing done for extracting the shadow 
of the pointer from the image-acquisition signal of 
the CCD camera and detecting the pointing posi- 
tion; 

Fig. 8A is an illustrative view of the step for extract- so 
ing the shadow of the pointer from the image-acqu i- 
sition signal of the CCD camera, and Fig. 8B is an 
illustrative view of specific image processing used 
for extracting the shadow of the pointer from the im- 
age-acquisition signal of the CCD camera; 55 
Fig. 9A is an illustrative view of the step of detecting 
the pointing position from the extracted shadow of 
the pointer, and Fig. 9B is an illustrative view of spe- 



cific image processing used for detecting the point- 
ing position from the extracted shadow of the point- 
er; 

Fig. 1 0A is an illustrative view of binary image data 
of the shadow region of a pointing image that is fac- 
ing in the lateral direction, Fig. 1 0B is an illustrative 
view of binary image data of the shadow region of 
a pointing image that is facing in the vertical direc- 
tion, and Fig. 10C is an illustrative view of binary 
image data of the shadow region of a pointing image 
that is facing in an oblique direction; 
Fig. 1 1 A is an illustrative view of an ordinary cursor 
display based on the pointing position, and Fig. 11 B 
is an illustrative view of cursor display near the pe- 
ripheral edge of the display region; 
Fig. 1 2A is an illustrative view of an example of dis- 
play in which the facing direction of the cursor is 
changed, based on the pointing position, and Fig. 
12B is an illustrative view of the algorithm for dis- 
playing the cursor facing sideways in the horizontal 
direction. 

Figs. 1 3A and 1 3B are illustrative views of shading 
countermeasures used in the present embodiment; 
Fig. 14A is an illustrative view of the processing for 
setting the binary-data threshold value V ref for ex- 
tracting the shadow of the pointer from the image- 
acquisition signal in an ideal state, Fig. 14B is an 
illustrative view of the processing for setting the bi- 
nary-data threshold value V ref for extracting the 
shadow of the pointer from the image-acquisition 
signal when there are brightness differences within 
the display region, and Fig. 14C is an illustrative 
view of the processing for setting the binary-data 
threshold value V ref for extracting the shadow of the 
pointer from the image-acquisition signal when ref- 
erence values are modified to improve the bright- 
ness differences; 

Fig. 15A is an illustrative view of the processing for 
dividing the display region into a plurality of regions 
and setting an individual reference value for each 
of these regions, and Fig. 15B is an illustrative view 
of an example of a calibration pattern image used 
when the reference values are set 
Fig. 16 is a flowchart of an operation of automati- 
cally setting the reference value when the system 
starts operating; 

Fig. 17 is a flowchart of the operation of resetting 
the reference value after the system has started op- 
erating; 

Fig. 18 is a detailed flowchart of the operation of 
automatically setting the reference value in the 
present embodiment; 

Fig. 19 is a detailed flowchart of the operation of 
displaying cursors which correspond to a plurality 
of pointers in the present embodiment; 
Fig. 20 is a detailed flowchart of the operation of 
fixing the display of the cursor position in the 
present embodiment; 
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Fig. 21 is a flowchart of the operation of changing 
the cursor size in the present embodiment; 
Fig. 22A is a block diagram illustrating the automatic 
setting operation of the optical control section of the 
CCD camera in the present embodiment; and s 
Fig. 23 is an illustrative view of the hardware con- 
figuration of the processing section in the present 
embodiment. 

DESCRIPTION OF THE PREFERRED EMBODIMENT 10 

[0112] A preferred embodiment of a presentation sys- 
tem to which the present invention is applied is de- 
scribed in detail below, with reference to the accompa- 
nying drawings. is 

1. Overall Description 

[011 3] An example of a presentation system that uses 
a front-projection type of projector apparatus as an im- 20 
age display apparatus is shown in Fig. 1. A predeter- 
mined image for a presentation is projected from a pro- 
jector 10 that is provided substantially facing a screen. 
A presenter 30 gives a presentation to spectators while 
using an indicator rod 40 to indicate desired positions of 25 
the image in the display region 12, that is a region on 
the screen on which the image is displayed. 
[0114] An example of the presentation given by the 
presenter 30 is shown in Fig. 4. 

[011 5] When the presenter 30 indicates a desired po- 30 
sition of the display region 12 on the screen using the 
indicator rod 40, as shown in Fig. 4A, the display region 
1 2, the indicator rod 40, and part of the presenter 30 are 
acquired as an acquired image 20 shown in Fig. 4B by 
a CCD camera 1 4, which is provided substantially facing 3S 
the display region 12 and which functions as an image- 
acquisition means. 

[0116] In this case, a long, narrow rod-shaped shad- 
ow that is formed by the indicator rod 40 as it interrupts 
the projected light from the projector 1 0 is called a shad- 40 
ow region 300 of the pointing image. An actual image of 
part of the presenter 30 and the indicator rod 40 that 
appear within the acquired image 20 is called a real-im- 
age region 302 and, when combined with information 
(target) used for detecting the pointing position, it is « 
called a detection target 304. 

[0117] The pointing position on the display region 12, 
which the presenter 30 indicates by using the shadow 
region 300 of the indicator rod 40, is detected as a tip 
position 310 of the shadow region 300 of the pointing so 
image that appears on the display region 12 in the ac- 
quired image 20. In other words, the tip position 310 of 
the shadow region 300 of the rod-shaped pointing image 
is automatically detected as the pointing position, and 
is subjected to predetermined data processing. ss 
[0118] In a similar manner, the pointing position on the 
display region 12, which the presenter 30 indicates by 
using the real-image region 302 of the indicator rod 40, 



is detected as a tip position 31 0 of the real-image region 
302 of the pointing image that appears on the display 
region 12 in the acquired image 20. 
[011 9] With the present embodiment, data processing 
is performed so that a perceived pointing position is in- 
dicated by a cursor 200 projected from the projector 1 0. 
In other words, in the display image projected onto the 
display region 1 2, the cursor 200 that is comprised with- 
in that image moves to follow the pointing position of the 
indicator rod 40. Thus the audience who are listening to 
the presentation of the presenter 30 are able to perceive 
the pointing position accurately and also easily, from the 
position of the cursor 200 displayed on the display re- 
gion 12. 

[01 20] Note that comer portions of the display region 
12, particularly the four comer portions thereof, has a 
brightness level that is lower than that of a central por- 
tion, as will be discussed later. To ensure that the shad- 
ow and actual image of the indicator rod 40 can be de- 
tected accurately, even in such regions, a light source 
1 6 is used to add supplementary lighting to those comer 
portions of the display region 12. 
[0121] An example of a rear-projection type of pres- 
entation system is shown in Fig. 2A. In this presentation 
system, the image from the projector 10 is projected 
from behind the display region 12, and that projected 
image is acquired by using the CCD camera 14 which 
is provided behind the display region 1 2 and which func- 
tions as image-acquisition means. The image from the 
projector 10 is projected via reflective plates 18-1 and 
1 8-2 onto the display region 1 2. 
[0122] Another example of a rear-projection type of 
presentation system is shown in Fig. 2B, wherein the 
image of the display region 12 is acquired by the CCD 
camera 14 that is provided on the outer side, substan- 
tially facing the screen. 

[0123] In this type of rear-projection system as well, 
the presenter 30 indicates a desired position on the dis- 
play region 1 2 with the indicator rod 40, and the shadow 
and actual image of that indicator rod 40 appear within 
the acquired image 20. The thus appearing shadow can 
be acquired by the CCD camera 14 as the shadow re- 
gion 300 of the pointing image and the actual image 
thereof can be acquired as the real-image region 302 of 
the pointing image. 

[01 24] The tip position 31 0 of the shadow region 300 
or the real-image region 302 of the pointing image is 
automatically perceived as the pointing position, and da- 
ta processing is applied thereto as described above. 
[0125] In addition to the projector 10 which is the 
above described projector type of display apparatus, 
various other types of display apparatuses could be 
used as the image display means of the rear-projection 
type of system to which the present invention is applied, 
such as a direct-viewing type of CFTT, a direct-viewing 
type of liquid crystal display panel, a plasma display 
panel, or other display apparatus. 
[0126] A specific example of the indicator rod 40 is 
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shown in Fig. 3. The indicator rod 40 used in the present 
embodiment is provided with a wide pointing portion 42 
at the tip of a rod-shaped portion thereof. This pointing 
portion 42 is made of a material having a low reflect- 
ance. Since light is not reflected from the tip of the indi- 
cator rod in this implementation, a clear detection target 
304 of the pointing image can be formed within the im- 
age-acquisition area and, as a result, it is possible to 
reliably detect the pointing position from the image-ac- 
quisition signal of the CCD camera 14. 
[0127] Note that in the front-projection type of system 
shown in Fig. 1, the presenter 30 can form the shadow 
region 300 of the pointing image within the acquired im- 
age 20 of the CCD camera 1 4 by positioning the indica- 
tor rod 40 in the light path between the projector 1 0 and 
the display region 12, without bringing the indicator rod 
40 into contact with the display region 12 directly. Sim- 
ilarly, the presenter 30 can form the real-image region 
302 of the pointing image within the acquired image 20 
by positioning the indicator rod 40 in the light path be- 
tween the display region 12 and the CCD camera 14, 
without bringing the indicator rod 40 into contact with the 
display region 1 2 directly. This means that the presenter 
30 can give the presentation from a position that is com- 
paratively far from the display region 12, while still being 
able to indicate any desired position on the display re- 
gion 12. 

[0128] A schematic functional block diagram of the 
system of the present embodiment is shown in Fig. 5. 
[0129] The system of the present embodiment com- 
prises an input source 1 00 that supplies an image to the 
projector 1 0 and a processing section 1 1 0 that automat- 
ically detects the pointing position from the image ac- 
quired by the CCD camera 14. Note that further details 
of the configuration of this processing section 110 will 
be given later. 

[0130] In the present embodiment, the projector 10 
and the CCD camera 14 are formed as an integral unit, 
but they could equally well be disposed separately. 
When they are disposed separately and a presentation 
is given with an enlarged screen image, particularly in 
a dedicated conference chamber or hall, this makes it 
possible to use existing equipment that is provided with- 
in the chamber, so that it can be freely modified in ac- 
cordance with the types of devices, their locations, and 
the surrounding environment, increasing the common- 
ality and convenience of use of the system. 
[0131] The configuration is such that the size and di- 
rection of images projected from the projector 1 0 can be 
freely set as required and the image-acquisition area of 
the CCD camera 14 can also be freely set as required. 
[0132] Polarized plates 10a and 14a are provided in 
the light paths of the projector 10 and the CCD camera 
1 4, respectively, to ensure that the CCD camera 1 4 can 
capture a clear image of the detection target 304 of the 
pointing image formed by the indicator rod 40 of the pre- 
senter 30. These polarized plates 10a and 14a are set 
in such a manner that the directions of polarization 



thereof are different from each other. This ensures that 
the CCD camera 14 is not greatly affected by the light 
emitted from the projector 10, thus enabling it to reliably 
acquire the detection target 304 of the pointing image 

s from the acquired image information. 

[01 33] The CCD camera 1 4 also has a zoom function 
that enables magnification or compression of the image- 
acquisition area, making it possible to free set this area 
to be greater or smaller than the display region 12. 

10 [01 34] Since this makes it possible to set the image- 
acquisition area freely in accordance with factors such 
as the usage environment or the image^acquisition cir- 
cumstances, it makes the system extremely convenient 
to use. When the image-acquisition area is set to be 

is greater than the display region, it will no longer be nec- 
essary to adjust the position of the image-acquisition ar- 
ea relative to the display region. 
[0135] In particular, it is possible to take the position 
of the presenter 30 into consideration and thus detect 

20 the pointing position in a more suitable manner, by set- 
ting the image-acquisition area so that it includes the 
presenter 30. 

[0138] The resolution of the CCD camera 14 can be 
increased substantially by setting the image-acquisition 
25 range to be smaller, so that the area specified for display 
on the display region 12 can be acquired at a higher res- 
olution to facilitate desired image processing. 



2. Detection of Pointing position 
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[0137] A detailed function block diagram of the 
processing section 110 is shown in Fig. 6. 
[01 38] In the present embodiment of the invention, the 
processing section 110 comprises a binary-data 

35 processing section 112, a pointing coordinates detec- 
tion section 116, and a computation section 118. This 
processing section 110 could be implemented by using 
components such as a CPU, ROM which is an informa- 
tion storage medium that contains programs and data, 

40 and RAM that functions as a work area. 

[0139] The image signal that is output from the CCD 
camera 14 is input to the binary-data processing section 
112. In the present embodiment, the CCD camera 14 is 
designed to output a monochrome image-acquisition 

45 signal. 

[0140] In the present embodiment, the binary-data 
processing section 1 1 2 compares the image-acquisition 
signal and a reference value V ref , then extracts the de- 
tection target 304 such as the shadow and actual image 
50 of the pointing image from the acquired image, and the 
pointing coordinates detection section functions as a po- 
sition detection means for detecting the pointing posi- 
tion. 

[0141] The binary-data processing section 112 also 
55 performs processing to compare brightness data of the 
image-acquisition signal that is output from the CCD 
camera 14 against the reference value V ref for binary 
data, and extract the detection target 304 of the pointing 
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image from within the image acquired by the CCD cam- 
era 1 4, then it outputs the processing data as binary im- 
age data to the pointing coordinates detection section 
116. 

[0142] The pointing coordinates detection section 1 1 6 
extracts the bulk of the detection target 304 from the 
binary image data that has been output from the binary- 
data processing section 112, detects the tip portion of 
this detection target 304 as the coordinates of the point 
indicated by the indicator rod 40, then outputs the result 
of this detection to the computation section 11 8. 
[0143] In the present embodiment, the pointing coor- 
dinates detection section 116 is configured in such a 
manner that the pointing image is specified on the basis 
of the continuous shape of the image of the detection 
target 304 extending in a rod shape, and a tip portion 
thereof is detected as the pointing coordinates. This 
makes it possible to increase the accuracy with which 
the pointing position is detected, in comparison with the 
case in which an angle of the image of the detection tar- 
get 304 is simply detected as the pointing coordinates. 
[0144] This is described in detail below. 
[0145] A flowchart illustrating the concept of the point- 
ing coordinates detection processing performed by the 
pointing coordinates detection section 116 is shown in 
Fig. 7A. In the present embodiment of the invention, the 
description particularly concerns detection processing 
in which the shadow region 300 of the pointing image is 
used as the detection target 304 of the pointing image, 
but the present invention is not limited thereto and thus 
a similar processing method can be applied for the real- 
image region 302 of the pointing image, by extracting a 
low-brightness region within the acquired image 20. 
[0146] First of all, processing is done to specify the 
shadow region 300 of the rod-shaped pointing image, 
from the binary image data that is output from the binary- 
data processing section 1 1 2. In other words, the shadow 
region 300 of the pointing image, which is the present- 
er's shadow 30a and the shadow of the indicator rod 40 
as shown atQin Fig. 7B, is comprised within the binary 
image data that is output from the binary-data process- 
ing section 112. 

[0147] In step S1 0, the pointing coordinates detection 
section 1 1 6 extracts the shadow region 300 of the point- 
ing image alone from the binary image data, as shown 
at©in Fig. 7B. This extraction processing is performed 
by extracting the shadow image portion that forms a 
continuous rod shape, from the binary-data image 
shown a©of Fig. 7B. 

[0148] The pointing coordinates detection section 116 
then performs the pointing position detection processing 
of step S12. In other words, this processing specifies 
where the pointing position is within the rod-shaped 
shadow region 300 of the pointing image that was ex- 
tracted at@of Fig. 7B. That is to say, processing is done 
to determine which of points A and B at either end of the 
shadow region 300 of the rod^shaped pointing image, 
as shown at@in Fig. 7B, is the pointing position. 



[0149] In this case, the binary image data that has 
been input from the binary-data processing section 112 
comprises the presenter's shadow 30a continuing from 
the B side of the shadow region 300 of the pointing im- 

s age, as shown at© in Fig. 7B. Thus point A, which is 
not connected to the presenter's shadow 30a, is speci- 
fied as the pointing position. This means that the coor- 
dinates tip of the shadow region 300 of the rod-shaped 
pointing image, in other words, the coordinates of point 

10 A, are detected as the pointing position that is output to 
the computation section 118. 

[0150] When binary image data that represents the 
shadow region shown in Fig. 7B is obtained by the bi- 
nary<Jata processing section 112, the processing in ac- 

is cordance with the present embodiment of the invention 
is such that first of all the rod-shaped shadow image por- 
tion is extracted as the shadow region 300 of the point- 
ing image, in step S10, then a specification is made as 
to which of the two ends of the thus extracted shadow 

20 region 300 of the pointing image is the pointing position, 
in step S1 2, and the coordinates of that pointing position 
are detected. This configuration makes it possible to in- 
crease the accuracy with which the pointing position is 
detected from the binary image data that is output from 

25 the binary-data processing section 112, in comparison 
with the case in which a comer portion of the shadow is 
specified as the pointing position. 
[0151] The processing of steps S10 and S12 is de- 
scribed in detail below. 

30 

2-1. Details of Step S10 

[0152] A specific example of the processing of step 
s1 0 that is shown in Fig. 8 is an example of the algorithm 

35 used for extracting the shadow region 300 of the point- 
ing image from the binary image data that is output from 
the binary-data processing section 112. 
[0153] In this case, the extraction of the shadow re- 
gion 300 of the rod-shaped pointing image is considered 

40 to be equivalent to the extraction of a long, narrow 
shape. 

[01 54] The long, narrow shadow is first detected, then 
the area of that long, narrow shadow is detected. A re- 
gion where the area is large is perceived to be continu- 
es ous, and that is extracted as the shadow region 300 of 
the pointing image. This is described in detail below. 
[0155] First of all, when binary image data 400 as 
shown in Fig. 8B is input from the binary-data process- 
ing section 112, the pointing coordinates detection sec- 
50 tion 116 starts the extraction of the shadow region 300 
of the pointing image (step S20). 
[01 56] This binary image data 400 comprises the pre- 
senter's shadow 30a and the shadow region 300 of the 
pointing image, which is the shadow of the indicator rod 
55 40, as previously described. 

[0157] HPF processing is performed in each of the 
horizontal and vertical directions on this binary image 
data 400, in steps S22 and S24. Here, HPF processing 
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means a processing that excludes shadow portions hav- 
ing at least a certain length, in both the horizontal and 
vertical directions. 

[0158] The images obtained by the processing of 
steps S22 and S24 are synthesized in step S26 to obtain s 
binary image data 402 from which the presenter's shad- 
ow 30a has been removed. In other words, only shadow 
images that have narrow portions in either the horizontal 
direction or the vertical direction remain, and all other 
shadow images have been removed. 10 
[0159] LPF processing, which is a processing that re- 
moves shadows having a certain length or less, is per- 
formed on the thus obtained binary image data 402 in 
step S2B, to obtain labeling-target binary image data 
404. This completely removes any fine noise 440. is 
[0160] Steps S30, S32, and S34 remove any detec- 
tion noise 450 in a region other than the shadow region 
300 of the pointing image, which was not removed by 
step S28. This removal of the detection noise 450 first 
involves the labeling processing in step S30, then an 20 
area is calculated for each label in step S32, and shad- 
ows greater than a certain area are extracted in step 
S34. 

[0161] In other words, a group of each of the detection 
noise 450 included within the labeling-target binary im- 2s 
age data 404 is separated for each region, these groups 
are labeled A to E in this figure, and thus labeling 
processing data 406 is obtained. 
[0162] The number of pixels is counted for each label 
(area). In practice, this can be done simultaneously with 30 
the labeling. This calculates the area of each shadow 
region for labels A to E. Any label with a small area is 
determined to be noise and is removed from the labeling 
processing data 406, to obtain binary image data 408 
that comprises only the shadow region 300 of the point- 35 
ing image. Note that step S34 could be such that it ex- 
tracts the label having the largest area. Alternatively, 
when a plurality of pointing positions are to be detected, 
as will be described later, it could extract the label with 
the largest area then the area having the next largest 40 
area. 

[0163] As an extraction method other than the area 
calculation of step S32, the shadow region 300 of the 
pointing image could be extracted from a continuous 
rod-shaped label. When the shape of the thus extracted *5 
labeling processing data 406, is a rod shape and it is 
also continuous, it can be identified as being the shadow 
generated by the indicator rod 40. However, when there 
is detect ion noise 450 that is small and is not contin uous, 
that is determined to be detection noise in the binary so 
image data that has not been removed by the various 
filters of steps S22, S24, and S28. The presenter's shad- 
ow 30a and noise are removed from the binary image 
data 400 that is output from the binary-data processing 
section 112 by this series of processes from step S20 ss 
to step S36, so that only the shadow region 300 of the 
long, thin pointing image is extracted thereby and thus 
the binary image data 408 can be obtained. 



[01 64] Note that when there is little noise in the binary 
image data 402 that has been subjected to HPF 
processing, step S28 will not be necessary. 

2-2. Details of Step S12 

[01 65] A specific algorithm illustrated in Fig. 9 is used 
for determining the pointing position from the shadow 
region 300 of the pointing image comprised within the 
binary image data 408 obtained by steps S20 to S36. 
[01 66] This processing searches for the most angular 
part of the shadow region 300 of the pointing image, as 
the pointing coordinates. Alternatively, even if there is a 
certain width at the tip of the shadow region 300 of the 
pointing image, such the most angular coordinates 
could be specified as the pointing coordinates. 
[01 67] When this processing starts (step S40), points 
A and B at the two longitudinal ends of the shadow re- 
gion 300 of the pointing image comprised within the bi- 
nary image data 408 are detected in step S42, as shown 
by(J)in Fig. 9B. A specific example of this detection will 
be described later with reference to Fig. 10. 
[0168] Next, step S44 determines whether or not the 
image of the shadow that is not the shadow region 300 
of the pointing image is in contact with or close to point B. 
[0169] When the answer is Yes, point B of the binary 
image data 400 that has been output from the binary- 
data processing section 112 (the binary image data be- 
fore it is subjected to the processing of steps S22 to S34) 
is connected to the presenters shadow 30a, as shown 
at ©in Fig. 9B. Therefore, in this case, point A at the 
opposite end is searched for as the pointing coordi- 
nates. 

[0170] Alternatively, when the answer obtained in 
step S44 is No, the next step S46 determines whether 
or not point B is at the edge of the screen image or in 
the vicinity thereof, such as in binary image data 410 
shown at@in Fig. 9B. ln@of Fig. 9B, the presenter 30 
is assumed to be pointing with a long indicator rod at a 
certain position on the display region from outside a 
screen image 41 0. In such a case, point B of the shadow 
region 300 of the pointing image is at the edge of the 
screen image, so point A at the opposite end thereof is 
searched for as pointing coordinates A. 
[0171] Alternatively, when the answer obtained in 
step s46 is No, the next step S48 determines whether 
point A is touching a shadow outside the shadow region 
300 of the pointing image, or is close thereto. In this 
case, when the answer is Yes, point B is searched for 
as the pointing position. When the answer is Yes in this 
case, point B is searched for as the pointing position; 
when it is No, a detection error is output. 
[0172] As described above, the processing of steps 
S44 and S46 of the present embodiment of the invention 
can detect the pointing position accurately when the pre- 
senter 30 is standing on the left side when facing the 
display region 12, and the processing of steps S48 and 
S50 can detect the pointing position accurately when the 
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presenter 30 is standing on the right side when facing 
the display region 1 2. In this manner, the processing de- 
termines which of points A and B at the two ends of the 
shadow region 300 of the pointing image is the pointing 
position, and detects the two-dimensional coordinates s 
(x, y) of that pointing position. 

2-3. Details of Step S42 

[0173] A specific example of the processing of step 10 
S42 of Fig. 9 is shown in Fig. 10. In this case, three sets 
of binary image data comprising the shadow region 300 
of the pointing image are shown. Fig. 10A shows the 
shadow region 300 in a lateral orientation, Fig. 10B 
shows it in a vertical orientation, and Fig. 10C shows it is 
in an oblique orientation. The shape and dimensions of 
the shadow region 300 of the pointing image vary in ac- 
cordance with the form of the indicator rod 40 that is 
used. The shape of the shadow region 300 of the point- 
ing image could be any of rounded, rectangular, or po- 20 
fygonal, by way of example, but the example used for 
the present embodiment of the invention is a long, thin 
oval shape. 

[0174] To search for the two ends of the shadow re- 
gion 300 of this pointing image, first of all the coordinates 25 
of the top left of the display region 12 are taken to be 
the origin (X, Y) = (0, 0), as shown in Fig. 10, then the 
x and y coordinates of the shadow region 300 of the 
pointing image within the image are checked to find the 
four smallest and largest points (the points having the 30 
most extreme coordinates, that is, the points that have 
no detected coordinates beyond them in any direction) 
are obtained as a, b, c, and d. 



a: (Xmin, Ya) 
b: (Xmax, Yb) 
c: (Xc, Ymin) 
d: (Xd, Ymax) 

[0175] If a, b, c, and d satisfy the condition: 



(Xmax - xmin) > (Ymax - Ymin) 

the points a and b are specified as points A and B 
at the two ends of the shadow region 300 of the pointing 
image in step 542. If a, b, c, and d satisfy the condition: 

(Xmax - Xmin) < (Xmax - Ymin) 

the points c and d are specified as points A and B 
at the two ends of the shadow region 300 of the pointing 
image. 

[0176] Therefore, if the shadow region 300 of the 
pointing image shown in Fig. 10 is taken as an example, 
the coordinates of a and b are detected as end points 
in the longitudinal direction in Figs. 10A and 10C, and 



the coordinates of c and d are detected as end points in 
the longitudinal direction in Fig. 10B. 
[0177] Note that the pointing coordinates detection 
section 116 of the present embodiment usually detects 
the latest indication position as the pointing position, 
based on time-series data of the indication using the in- 
dicator rod 40, then outputs the resultant detection po- 
sition data. 

[0178] Thus the pointing coordinates detection sec- 
tion 116 of the present embodiment detects the coordi- 
nates of the tip of the shadow region 300 of the pointing 
image as the pointing position and outputs data on that 
detection position to the computation section 118. 
[0179] Note that, although the above described em- 
bodiment extracts the portion of the indicator rod 40 that 
casts a shadow in the acquired image 20 on the display 
region 12 as the shadow region 300 of the pointing im- 
age, and detects the pointing position therefrom, other 
indication portions such as the shadow cast by the finger 
of the presenter 30 could equally well be perceived as 
the shadow region 300 of the pointing image, so that the 
tip position of the finger could be automatically per- 
ceived as the pointing position. 
[01 80] ^ addition, although the above described em- 
bodiment outputs a monochrome image signal from the 
CCD camera 14 and uses the binary-data processing 
section 112 and the pointing coordinates detection sec- 
tion 1 1 6 to separate the shadow region 300 of the point- 
ing image from this image signal and thus detect the 
pointing position, a method similar to that of the above 
embodiment could be used even if a color image-acqui- 
sition signal is output from the CCD camera 14, such 
that the brightness level of the image-acquisition signal 
is compared with a reference value V ref to extract the 
shadow of the pointing image and thus detect the point- 
ing position. 

2-4. Shading Counter-measures 



40 [0181] Depending on the environment in which the 
presentation system is used, it can happen that a low- 
brightness region 320a having a brightness level lower 
than that of other regions is formed in a part of the image 
projected onto the display region 1 2, particularly in a cor- 

45 ner portion thereof. If this image is acquired by the CCD 
camera 14 and subjected to binary-data processing by 
the binary-data processing section 112, a low-bright- 
ness region 320b shown in Fig. 1 3B will be extracted as 
a shadow. This makes it difficult to detect the pointing 

so position accurately, particularly when the shadow region 
300 of the pointing image overlays the tow-brightness 
region 320b. 

[0182] To solve this problem, the system of the 
present embodiment of the invention uses an il lumina- 
ls tion means 1 6 consisting of a light or LED array, to con- 
centrate the lighting on the low-brightness region 320a 
having a brightness level that is low within the display 
region 12. This ensures that the portion within the bina- 
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ry-data image processed by the binary-data processing 
section 1 1 2 that corresponds to the low-brightness area 
320a is removed, making it possible to extract the shad- 
ow region 300 of the pointing image easily from this bi- 
nary image data and thus detect the pointing position. s 

3. Data Processing Based on Detected Position Data 

[0183] The computation section 118 shown in Fig. 6 
performs data processing and image processing, based 10 
on detection data for the pointing position that is input 
thereto. 

[0184] In the present embodiment of the invention, 
this computation section 1 1 8 functions as a camera con- 
trol section 1 22, a cursor control section 1 20, and a au- is 
tomatic reference value setting section 114. 

3-1. Details of Camera Control Section 122 

[01 85] Trie camera control section 1 22 performs var- 20 
bus kinds of optical control, such as focus control for 
the CCD camera 14, based on information that is input 
from an operator control section 1 30 or a projector (PJ) 
optical control section 150 of the projector 10. The de- 
scription now turns to optical control based on informa- 25 
tion that is input from the PJ optical control section 150 
of the projector 10, in particular. 
[01 86] A method shown in Fig. 22 provides automatic 
adjustment of the optical system of the image-acquisi- 
tion lens of the CCD camera 14, linked to various ad- 30 
justments such as focus and zoom in the projection lens 
of the projector 10. 

[01 87] A block diagram illustrating the overall config- 
uration is shown in Fig. 22A. 

[0188] Ordinarily, the presenter 30 manually adjusts 35 
a PJ lens adjustment mechanism 152 that is an optical 
adjustment mechanism for the projection lens of the pro- 
jector 10 to ensure that the focus of the display image 
projected from the projector 10 is optimized, or an au- 
tomatic adjustment mechanism is used to adjust it au- 40 
tomatically. The amount of adjustment of this PJ lens 
adjustment mechanism 152 is detected by a sensor 154 
and is input to the camera control section 122. 
[0189] The camera control section 122 executes a 
previously written lens control program 1 26 for control 45 
ling the amount of optical adjustment of the CCD lens, 
based on the detection data from the sensor 154. The 
execution result is supplied to a CCD optical control sec- 
tion 22 within the CCD camera 14, and this CCD optical 
control section 22 controls a CCD lens adjustment so 
mechanism 24 that adjusts the image-acquisition lens. 
Th is makes it possible to link the control of the projection 
lens of the projector 10 to the adjustment of the image- 
acquisition lens of the CCD camera 14. 
[01 90] Although it was necessary in the prior art to ad- ss 
just the lenses of the projector 1 0 and the CCD camera 
14 separately, the configuration of the present embodi- 
ment allows the presenter 30 to adjust the projection 



lens of the projector 10 manually or activate an opera- 
tion that enables automatic adjustment to perform fine 
optical adjustment of the CCD camera 14 automatically, 
thus making it possible to remove the troublesome labor 
involved with fine optical adjustment of the CCD camera 
14, improving the operability and ease of use of the sys- 
tem. 

[0191] The flow of the processing shown in Fig. 22A 
is shown in Fig. 22B in algorithm form. 
[01 92] First of all, the PJ lens adjustment mechanism 
152 is manipulated by the user in step S202, and the 
amount of that adjustment is detected by the sensor 1 54 
(step S204). The amount of control in the CCD lens ad- 
justment mechanism 24 is searched for by the lens con- 
trol program 126, based on the amount of adjustment 
detected by the sensor 1 54 (step S206), and the adjust- 
ment mechanism of the CCD lens is automatically con- 
trolled in step S208. 

[01 93] The cursor control section 1 20 controls the po- 
sition of the cursor 200 in such a manner that it indicates 
the detected pointing position. In other words, the input 
source 1 00 is controlled in such a manner that the cursor 
200 comprised within the image projected from the pro- 
jector 1 0 follows the pointing position of the indicator rod 
40. 

[01 94] The automatic reference value setting section 
114 performs processing for automatically setting to an 
optimal value a reference value V ref for binary data used 
by the binary-data processing section 112. 
[0195] The description now turns to details of the cur- 
sor control section 120. 

3-2. Details of Cursor Control Section 120 

[01 96] The cursor control section 1 20 controls the po- 
sition of the cursor 200 in such a manner that it indicates 
the detected pointing position, at a position which does 
not overlay the shadow region 300 of the pointing image. 
[01 97] An example of positional control of the cursor 
200 is shown in Fig. 11A. In this example, the display 
position of the cursor 200 is controlled to a position that 
is a fixed distance away along a line extending from the 
shadow region 300 of the pointing image. More specif- 
ically, the continuous shape of the shadow region 300 
of the pointing image, which is the shadow of the indi- 
cator rod 40, is perceived and the cursor 200 is control- 
led so that it is positioned on a line extending therefrom. 
[0198] During this time, the cursor 200 appears at a 
position such that it does not overlay the indicator rod 
40, such as at a position that is separated from the tip 
of the indicator rod 40 by one dimension of the cursor. 
[0199] This ensures that the cursor 200 is not con- 
cealed by the shadow of the indicator rod 40, and the 
pointing position of the presenter 30 can be indicated 
reliably and in an easy-to-see manner by the cursor 200. 
[0200] The cursor control section 120 could also be 
configured in such a manner that details of the cursor 
200, such as the size, shape, and color thereof, are con- 
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trolled in accordance with the thickness of the tip of the 
shadow region 300 of the pointing image. If, for exam- 
ple, the shadow region 300 of the pointing image is nar- 
row, the position of the tip thereof would be difficult to 
identify accurately, so the pointing position can be made s 
easier for the audience to see by displaying the cursor 
200 large. 

[0201] If the cursor 200 is displayed on a line extend- 
ing from the indicator rod 40, as shown in Fig. 11 A, it is 
possible that the cursor 200 will be concealed by the 10 
peripheral edge portion of the display region 12 or it 
might appear in an area that cannot be displayed. 
[0202] For that reason, when the pointing position ap- 
proaches the peripheral edge portion of the display 
screen and thus a fixed distance = d can no longer be is 
maintained along the line extending from the shadow 
region 300 of the pointing image, as shown at©in Fig. 
11B, the cursor 200 could be displayed closer to the 
pointing position (as shown at® in this figure) or over- 
laying the pointing position (as shown at@in this figure). 20 
The cursor 200 could also be controlling in such a man- 
ner that a detail thereof, such as its size, color, or shape, 
varies in accordance with the area by which the cursor 
200 overlaps the tip of the shadow region 300 of the 
pointing image. In this figure, ©shows an example in 2s 
which the size and shape of the cursor are varied and 
©shows an example in which the color thereof is varied. 
This ensures that the cursor 200 does not merge with 
the shadow of the indicator rod 40 and become invisible 
to the audience, even at the peripheral edge portion of 30 
the display screen, and also makes the pointing position 
easier to see. 

[0203] The algorithm used when changing the cursor 
size as described above is shown in Fig. 21 . 
[0204] First of all, in step S162, a length d and a width 35 
t are calculated as part of the area calculation of step 
S32 of Fig. 8. In the present embodiment of the inven- 
tion, the shaft portion of the indicator rod 40 is of a dif- 
ferent shape from the tip portion thereof, but the shaft 
portion of the indicator rod 40 could equally well be of 40 
the same shape as the tip portion. In this case, a length 
d 1 corresponding to the tip portion of the indicator rod 40 
is cut off, so that the tip shape can be considered to have 
a length d* and a width 1 

[0205] The length d in the longitudinal direction of the 45 
detection target 304 of the detected pointing image is 
then compared with a previously determined freely set- 
table length L (step S1 64). If d < L, the tip shape is de- 
termined to be smaller than the setting so that the size 
of the cursor 200 is made larger. If d > L, on the other so 
hand, the tip can be determined to be sufficiently large. 
[0206] Next, step S166 compares the width t of the 
detection target 304 of the detected pointing image with 
a previously set width k. If t < k, the tip shape is deter- 
mined to be smaller than the setting, in a similar manner 55 
to the length comparison, and the size of the cursor 200 
is changed to be larger. If t > k, on the other hand, the 
tip is determined to be sufficiently large. 



[0207] This processing is done each time that step 
S34 is executed, enabling real-time cursor control. 
[0208] The cursor control section 1 20 could also con- 
trol the display position of the cursor 200 as shown in 
Fig. 12A. 

[0209] If the presenter 30 gives the presentation while 
standing on the left side facing the display region 1 2, for 
example, the display region 1 2 can be divided into a nar- 
row region 220 along the right edge and a remaining 
region 230, and the narrow region 220 along the right 
edge can be further divided into a region 220a on the 
upper side and a remaining region 220b on the lower 
side. If the pointing position is in the region 230 that ex- 
cludes the right edge, the cursor 200 is displayed orien- 
tated horizontally to the left, on the same level as the 
pointing position. If the pointing position is in the upper- 
side region 220a on the right edge, the cursor 200 is 
displayed orientated upward on the lower side of the 
pointing position. If the pointing position is in the lower- 
side region 220b excluding the upper side on the right 
edge, the cursor 200 is displayed orientated downward, 
on the upper side of the pointing position. This makes it 
possible to prevent the occurrence of a state that can 
cause the cursor 200 to become concealed by the right 
edge of the screen image when the presenter gives a 
presentation while standing on the left side of the display 
region 12. 

[0210] Similarly, when the presenter gives a presen- 
tation while standing at the right edge of the display re- 
gion 12, the above described regions 220a and 220b 
could be provided on the left edge of the projection area 
and the display of the cursor 200 can be adjusted in a 
similar manner. In that case, the cursor is preferably dis- 
played orientated horizontally to the right, on the same 
level as the pointing position, mainly in the region 230. 
[0211] When the display of thecursor200 iscontrolled 
as shown in Fig. 12A, the cursor control section 120 
could be configured in such a manner as to enable 
switching of the direction of display of the cursor, de- 
pending on whether the presenter 30 is standing on the 
right side or the left side of the display region 12, by an 
input from the operator control section 130. 
[0212] This makes it possible to implement a presen- 
tation system that is easy to use. 
[021 3] A specific example of the algorithm used to dis- 
play the cursor at a fixed distance in the horizontal di- 
rection from the tip coordinates of the shadow region 
300 of the pointing image, as shown atQin Fig. 1 2A, is 
shown in Fig. 12B. 

[021 4] The algorithm for displaying the cursor is start- 
ed in step S100 and a step S102 detects the tip coordi- 
nates (x, y) of the pointing position. 
[0215] Step S104 then calculates the tip coordinates 
(x 1 , y 1 ) of the cursor, based on the detected coordinates 
(x, y), to determine the cursor coordinates (x\ y') (St 06). 
[021 6] Step S1 08 performs processing for controlling 
the cursor so that it is displayed at those cursor coordi- 
nates (x*, y'), to enable the display of the cursor at the > 
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desired position on the display screen image. 
[0217] The description now turns to a method by 
which a plurality of pointing positions is detected and the 
display of a plurality of cursors is controlled independ- 
ently, with reference to Fig. 1 9. s 
[021 8] First of all, step S1 22 performs the processing 
to extract the detection target 304 of the pointing posi- 
tion in step 534 of Fig. 8, and step S124 checks the 
number of extracted labels. In this case, ordinary 
processing for displaying a single cursor can be done if f o 
the number of extracted labels is one. If detection tar- 
gets 304 are extracted for a plurality of pointing images 
in step S124, the number thereof is calculated in step 
S126 and processing to display the equivalent number 
of cursors is done in step S1 28. is 
[021 9] This series of processes is repeated to display 
a cursor at each pointing image position. 
[0220] A specific algorithm illustrated in Fig. 20 is 
used when a plurality of cursors is displayed by the 
method described above, where one of these cursors is 20 
to be fixed at the position at which the cursor is displayed 
and the position of another cursor is to be controlled 
freely. 

[0221] The presenter 30 uses the indicator rod 40 to 
indicated a desired position on the display region 1 2, as 2s 
shown at®of Fig. 20, and a first cursor 202 is displayed 
at that pointing position (steps S142 and S144). 
[0222] The presentation system of the present em- 
bodiment is provided with a function switch 1 40 to which 
a predetermined function is allocated, as will be de- 30 
scribed below. Operating this function switch 1 40 sends 
an instruction to temporarily halt the motion of the first 
cursor 202 (step S146) so that the display position of 
the first cursor 202 is fixed as shown at®in Fig. 20 (step 

S148). 35 

[0223] Next, as shown at@in Fig. 20, a point that is 
not the previous point on the display region 12 is indi- 
cated, and a second cursor 204 is displayed at the new 
pointing position (steps S150 and S152). 
[0224] This means that the old and new pointing po- 40 
sitions can be indicated simultaneously and also sepa- 
rately. In particular, it is preferable to perform processing 
to distinguish the previously displayed first cursor 202 
from the second cursor 204 that is displayed thereafter, 
by making the color or shape of the second cursor 204 
different, or flashing the display. 
[0225] The configuration could be such that the first 
cursor 202, which is displayed fixed remains fixed at that 
display position until the presenter 30 operates the func- 
tion switch 140 in step S154, or the fixed display could so 
be released after a certain period of time has elapsed. 
The first cursor 202 that is released from the fixed dis- 
play in this manner is not detected as a pointing image 
on the display region 12, so it disappears from the 
screen image as shown at@of Fig. 20. The second cur- ss 
sor 204, which is displayed continuously, on the other 
hand, is displayed so that it follows the indication on the 
screen image, and the second cursor 204 is handled 



subsequently as if it is the first cursor 202. 
[0226] Any function than the above described func- 
tions that can be activated in a similar manner by a left- 
click or right-click of the mouse can be allocated to the 
function switch 140. 

[0227] This function switch 140 could be provided in- 
tegrally with the indicator rod 40, or it could be provided 
as a separate unit that the presenter 30 can carry freely. 
[0228] The above described configuration makes it 
possible for the presenter 30 to use the indicator rod 40 
to guide the cursor 200 to a desired position, then cause 
the cursor control section 120 to perform processing 
similar to that obtained by left-click or right-click of the 
mouse, by operating the function switch 140. 
[0229] It is also possible to draw an underline at a de- 
sired position of the image displayed on the display re- 
gion 1 2 or input handwriting by moving the pointing po- 
sition, by allocating an underline function or handwriting 
input function to the function switch 140. 
[0230] Furthermore, allocating a temporary-halt func- 
tion to the movement of the cursor 200 makes it possible 
for the presenter 30 to use the indicator rod 40 to move 
the cursor to a desired position then operate the function 
switch 140 to fix the display position of the cursor 200 
until the function switch 140 is operated again. 

4. Automatic Setting of Reference value 

[0231] The description now turns to details of the ref- 
erence value V ref set by the automatic reference value 
setting section 114. 

[0232] To ensure that the shadow region 300 of the 
pointing image is extracted reliably from the image-ac- 
quisition signal that is output from the CCD camera 1 4, 
it is necessary to set the reference value V ref , which is 
the threshold value for binary data, to the optimum val- 
ue. 

[0233] This reference value must be set to a value that 
is less than the lowest brightness level comprised within 
an image-acquisition signal that is created when the 
CCD camera 1 4 has acquired a picture of the image pro- 
jected onto the display region 12. If a monochrome im- 
age-acquisition signal is output from the CCD camera 
14, as in the present embodiment by way of example, 
the reference value must be set to a value that is less 
than that of the black brightness level. This ensures that 
the brightness level of the shadow region 300 of the 
pointing image is lower than the brightness level of the 
black color that has the bwest brightness level on the 
screen image. 

[0234] The brightness levels of each area obtained by 
the acquisition of an image by the CCD camera 14 are 
shown in Fig. 14A 

[0235] In this case, a completely white image (a dis- 
play-white region, shown at0in this figure) and a pitch 
black image (a display-black region, shown at ©in this 
figure) are projected on the display region 12 from the 
projector, and a state is also formed in which part of the 
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display region 1 2 is covered so that the projected image 
is not displayed on part of the display region 12 (a real 
black area, shown at@in this figure). Fig. 14A shows 
the brightness level for each region, obtained by the 
CCD camera 14 acquiring the resultant image. A shad- 
ow cast by the indicator rod 40 (a detected-black region, 
shown at@in this figure) has a brightness level (300a) 
that lies between a brightness level of image black 
(VLC) and a brightness level of real black (VB). Thus, 
to extract the shadow region 300 of the pointing image 
reliably, it is necessary to set the reference value V ref 
for binary data to a value that is less than the brightness 
level of image black (VLC) but greater than the bright- 
ness level of real black (VB). Note that the brightness 
levels in this figure are expressed as brightnesses levels 
in one horizontal scanning line of the CCD camera 14. 
[0236] Note that Fig. 14A assumes an ideal display 
region environment and shows the brightness level of 
each region obtained when the image projected on the 
display region 1 2 is acquired under such conditions, but 
the brightness levels obtained by the CCD camera 14 
from the image projected onto the display region 12 in 
practice are shown by way of example in Fig. 14B. Fig. 
14B shows that the brightness levels of the image pro- 
jected onto the display region 12 are fairly high at the 
center of the display region (shown at ©in this figure) 
but become lower nearer the edges of the display region 
(shown at©in this figure). A similar state can occur if 
there is a window near the display region 12 or the in- 
door lighting environment varies. 
[0237] Therefore, if the level difference AV (hereinaf- 
ter called margin) between the black level 300a of the 
shadow region 300 of the pointing image and the refer- 
ence value V ref is set too low, a situation will occur in 
that it is not possible to detect the shadow region 300 of 
the pointing image reliably at the center. In other words, 
the binaryKJata processing output of Fig. 14B shows 
that, if there are large variations in the brightness level 
300a of the shadow region 300 of the pointing image at 
the center (at©in this figure), this brightness level 300a 
of the shadow region 300 will not fall as far as below the 
reference value V ref . In such a case, it will not be possi- 
ble to extract this shadow region 300 reliably. 
[0238] For that reason, the automatic reference value 
setting section 1 1 4 of the present embodiment of the in- 
vention is configured to enable automatic setting of the 
reference value V ref to the optimum value. 
[0239] This is described in detail below 
[0240] A state shown in Fig. 15A is that of an image 
projected onto the display region 12 that has been di- 
vided into a plurality of regions, which is nine regions in. 
this case. 

[0241] The automatic reference value setting section 
1 1 4 of the present embodiment can set an individual ref- 
erence value V ref for each of these partial areas. 
[0242] For example, the reference value is set to 
v ref 1 « v ref2> and Vref 3 for partial areas 1, 2, and 3, as 
shown in Fig. 14C. , 



[0243] This ensures that the shadow region 300 of the 
pointing image can be detected reliably even at the cen- 
tral portion of the screen image, where the brightness 
level is high (the portion shown at©in Fig. 14B). 
s [0244] To enable the setting of an individual reference 
value V ref for each partial area, a calibration pattern im- 
age generation section 124 in accordance with the 
present embodiment of the invention causes a prede- 
termined calibration pattern image from the projector 10 
10 onto the screen. At the same time, the automatic refer- 
ence value setting section 114 verifies the data of the 
projected calibration pattern image and the image-ac- 
quisition signal acquired by the CCD camera 14, and 
sets the optimal reference value V ref for each partial ar- 
ts ea. 

[0245] In this case, the calibration pattern image 
shown in Fig. 15B is configured in such a manner that 
it is projected onto the display region 1 2. This calibration 
pattern image 500 comprises nine areas 500-1, 

20 500-2, ... 500-9 corresponding to the partial areas, 
where each area has a black area 510 at the center 
thereof, with a white area 512 disposed therearound. 
Thus the reference level of each partial area can be set 
to the optimum value from this calibration pattern image 

25 500, by the disposition of a pattern in which areas 510 
and 512 of different brightness levels, for white and 
black, in each area 500-1, 500-2, ... 500-9. 
[0246] Note that thecalib ration pattern image project- 
ed from the projector during this setting of reference val- 

30 ues could be configured such that black, which has the 
lowest brightness level, forms the reference. It is ateo 
possible to project a calibration pattern image that has 
a completely black level over the entire area of the cal- 
ibration pattern image. 

35 [0247] However, even with the same black, the bright- 
ness level of black in the image projected onto the dis- 
play region 12 and acquired by the CCD camera 14 will 
differ slightly between a case in which the surroundings 
are completely black and a case in which the surround- 

40 ings are completely white. 

[0248] The present embodiment uses a calibration 
pattern formed of a combination of the black area 510 
and the white area 51 2 having different brightness levels 
is set for each area 500-1 , 500-2, ... 500-9 and projects 

45 that combination image as the calibration pattern image 
500 onto the display region 12, so that each reference 
value can be set to the optimum value even in the above 
cases. This makes it possible to automatically set the 
reference value for binary data to the optimum value for 

50 each partial area. 

[0249] A flowchart of the automatic setting of the ref- 
erence values when the presentation system starts op- 
erating is shown in Fig. 1 6. The automatic reference val- 
ue setting section 1 1 4 starts the automatic setting of ref- 

55 erence values when the system starts operating (step 
S60). First of all, the turning on of an image projection 
switch of the projector 10 is detected (step S62), then 
the calibration pattern image shown in Fig. 15B is pro- 
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jected from the projector 10 onto the display region 12 
(step s64). 

[0250] The automatic reference value setting section 
114 verifies previously determined data for the bright- 
ness levels of the calibration pattern image against the s 
brightness level of the position that corresponds to the 
image-acquisition signal that is output from the CCD 
camera 14, and sets the optimal reference value V ref for 
each partial area (step S66). 

[0251] In this case, the signal brightness level corre- io 
spending to the black area 51 0 of the calibration pattern 
image is read from the image-acquisition signal for each 
area, and a reference value V ref having a predetermined 
margin with respect to that brightness level is set for 
each partial area. is 
[0252] After this setting has ended, the automatic ref- 
erence value setting section 114 ends the projection of 
the calibration pattern image and switches the image 
output of the projector 1 0 to ordinary image input mode 
(step S68), then ends the automatic reference value set- 20 
ting operation (step S70). 

[0253] In this manner, the present embodiment 
makes it possible to automatically set the threshold val- 
ue V ref for binary-data processing to the optimum value 
for each partial area when the system starts operating. 25 
[0254] A flowchart for resetting the reference value 
V ref after the system has started operating is shown in 
Fig. 17. 

[0255] When the user operates a reference value re- 
set switch 132, the automatic reference value setting 30 
section 114 detects the operation of this reset switch 1 32 
(step s82) and starts the reference value reset operation 
of steps S84 to S90. 

[0256] Note that steps S84 to S88 are similar to those 
of steps S64 to S68 of Fig. 1 6, so detailed description 35 
thereof is omitted. 

[0257] Thus the present embodiment makes it possi- 
ble to automatically set the reference values to optimum 
values even after the system has started operating, so 
that, if the usage environment has changed during the 40 
presentation (for example, when the environment out- 
side a window has become brighter or darker, or the 
brightness level on the display region 12 has changed), 
the user can set the reference value V ref for binary-data 
processing to the optimum value for the current usage 45 
environment by pressing the reference value reset 
switch 132. 

[0258] A specific algorithm shown in Fig. 18 illustrates 
details of the steps S64 and S66 of Fig. 1 6 and the steps 
S84 and S86 of Fig. 17. so 
[0259] First of all, step S 1 82 selects whether the ref- 
erence value V ref is to be preset or whether the previ- 
ously used reference value V mf is to be used without 
change. 

[0260] When the usage environment is exactly the ss 
same as it was previously, or is similar to that environ- 
ment, No can be selected to use the previously used 
reference value V ref . This is convenient in that it makes 



it possible to omit the reference value setting. 
[0261] When, on the other hand, Yes is selected in 
step S1 82 to preset the reference value V ref> the number 
of areas for which reference values are to be set is spec- 
ified (step S184) and also the size thereof is specified 
(step S186). This makes it possible to determine the ar- 
eas shown in Fig. 15A. Next, the type of calibration pat- 
tern, such as that shown in Fig. 15B, is selected in step 
S188. 

[0262] Brightness data is then detected for each par- 
tial area (step S190), and the thus detected brightness 
data is compared with pre-stored brightness data in step 
S192. Based on the results of this comparison, the ref- 
erence value V ref is controlled to a desired value (step 
S1 94) and that value is stored (step S196). 
[0263] The processing of steps S64 and S66 of Fig. 
1 6 and of steps S84 and S86 of Fig. 1 7 ends in step 1 98. 
[0264] In this manner, the present embodiment of the 
invention automatically sets each reference value V ref 
to the optimum value, making it possible to extract the 
shadow region 300 of the pointing image accurately 
from the image-acquisition signal. 
[0265] The present embodiment of the invention was 
described as involving the automatic setting of a differ- 
ent reference value for each area, by way of example, 
but the configuration could also be such that a single 
reference value is set automatically for all areas, if nec- 
essary. 

5. Hardware Configuration of Processing Section 

[0266] The description now turns to an example of the 
configuration of hardware that can implement this 
processing section 110, with reference to Fig. 23. In the 
apparatus shown in this figure, a CPU 1000, ROM 1002, 
RAM 1004, an information storage medium 1006, an im- 
age generation IC 1010, and I/O ports 1020-1, 
1020-2, ... are connected together by a system bus 
1016 in such a manner that data can be mutually trans- 
ferred therebetween. The CCD camera 14, the function 
switch 140, the operator control section 130, and the 
projector 10 are connected to other equipment by the I/ 
O ports 1020-1, 1020-2,.... 

[0267] Programs and image data are mainly stored in 
the information storage medium 1006. 
[0268] The CPU 1000 controls the entire apparatus 
and processes data in accordance with the programs 
stored in the information storage medium 1 006 and pro- 
grams stored in the ROM 1 002. The RAM 1 004 is a stor- 
age means that is used as a work space for the CPU 
1000, and specific details from the information storage 
medium 1 006 or the ROM 1 002, or the results of calcu- 
lations by the CPU 1 000, are stored therein. A data con- 
figuration having a logical structure suitable for imple- 
menting the present embodiment of the invention is con- 
structed within this RAM or information storage medium. 
[0269] The processing described with reference to 
Figs. 1 to 7, 10, 11 , and 13 to 15 is implemented by com- 
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ponents such as the information storage medium 1006 
that contains programs for performing processing such 
as that shown in the flowcharts of Figs. 8, 9, 1 2, and 1 6 
to 22, the CPU 1000 that operates in accordance with 
those programs, and the image generation IC 1010. 
Note that the processing performed by the image gen- 
eration IC 1010 could be performed in a software man- 
ner by means such as the CPU 1000 or an ordinary DSP. 
[0270] Note that the present invention is not limited to 
the previously described embodiments, and thus many 
modifications thereto are possible within the range of the 
invention as disclosed herein. 

[0271] For example, a CCD camera or the like that 
acquires a picture as a color image could be used, if 
necessary. 

[0272] In addition, the method of extracting the real- 
image region 302 of the pointing image could be such 
that the movement of the arm of the presenter 30 and 
the indicator rod 40 are detected, instead of the previ- 
ously described method in which the brightness level is 
detected on the basis of an image-acquisition signal. 



Claims 

1 . A pointing position detection device comprising: 

image-acquisition means for acquiring an im- 
age of a display region in which an image is dis- 
played; and 

position detection means for detecting a point- 
ing position from an image region of a pointing 
image included within said display region, 
based on an image-acquisition signal of said 
image-acquisition means. 

2. The pointing position detection device as defined in 
claim 1, wherein said position detection means de- 
tects as said pointing position a predetermined po- 
sition of a shadow region of said pointing image in- 
cluded within said display region. 

3. The pointing position detection device as defined in 
claim 1 , wherein said position detection means de- 
tects as said pointing position a predetermined po- 
sition of a real-image region of said pointing image 
included within said display region. 

4. The pointing position detection device as defined in 
any one of claims 1 to 3, wherein said position de- 
tection means captures a rod-shaped image region 
as said image region of said pointing image, based 
on the continuous shape of said image region in- 
cluded within said display region, and detects a tip 
position of said image region as said pointing posi- 
tion. 

5. A pointing position detection device comprising: 
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image-acquisition means for acquiring an im- 
age of a display region in which an image is dis- 
played; 

reference-value setting means for setting a ref- 
erence value for separating an image region of 
a pointing image included within said display re- 
gion, from an image-acquisition signal of said 
image-acquisition means; and 
position detection means for detecting a point- 
ing position from said image region of said 
pointing image, by comparing said image-ac- 
quisition signal and said reference value and 
thus extracting said image region of said point- 
ing image, 

wherein said reference-value setting means is 
configured to be capable of modifying said ref- 
erence value. 

6. A presentation system comprising: 

image display means for displaying an image; 
image-acquisition means for acquiring an im- 
age of a display region in which an image is dis- 
played; 

position detection means for detecting a point- 
ing position from an image region of a pointing 
image included within said display region 
based on an image-acquisition signal of said 
image-acquisition means; and 
cursor control means for controlling a position 
of a cursor included within the image displayed 
by said image display means, based on the de- 
tected pointing position. 



& 7. The presentation system as defined in claim 6, 
wherein said position detection means captures a 
rod-shaped image region as said image region of 
said pointing image, based on the continuous 
shape of said image region included within said dis- 
play region, and detects a tip position of said image 
region as said pointing position. 

8. A presentation system comprising: 

image display means for displaying an image; 
image-acquisition means for acquiring an im- 
age of a display region in which an image is dis- 
played; 

reference-value setting means for setting a ref- 
erence value for separating an image region of 
a pointing image included within said display re- 
gion from an image-acquisition signal of said 
image-acquisition means; 
position detection means for detecting a point- 
ing position from said image region of said 
pointing image, by comparing said image-ac- 
quisition signal and said reference value and 
thus extracting said image region of said point- 
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ing image; and 

cursor control means for controlling a position 
of a cursor included within the image displayed 
by said image display means, based on said 
detected pointing position, s 
wherein said reference-value setting means is 
configured to be capable of modifying said ref- 
erence value. 

9. The presentation system as defined in claim 8, fur- 10 
ther comprising: 

calibration pattern image generation means for 
displaying a predetermined calibration pattern 
image from said image display means to said is 
display region, 

wherein said reference-value setting means 
automatically sets a reference value for extract- 
ing said image region of said pointing image, 
based on data of said calibration pattern image *o 
and said image-acquisition signal. 

10. The presentation system as defined in claim 9, 
wherein said reference-value setting means divides 
said display region into a plurality of regions and 2s 
sets an individual reference value for each of the 
divided regions. 

11. The presentation system as defined in claim 9 or 

10, wherein said calibration pattern image genera- 30 
tion means displays a predetermined calibration 
pattern image formed by combining a plurality of re- 
gions having different brightnesses on said display 
region. 

35 

12. The presentation system as defined in any one of 
claims 9 to 11, further comprising: 

indication means for setting a reference value, 
wherein, when a reference-value setting in- 40 
struction is input by said indication means: 
said calibration pattern image generation 
means causes the display of a calibration pat- 
tern image from said image display means on 
said display region; and 45 
said reference-value setting means automati- 
cally sets a reference value for extracting said 
image region of said pointing image, based on 
data of said calibration pattern image and said 
image-acquisition signal. so 

13. The presentation system as defined in any one of 
claims 6 to 12, wherein said cursor control means 
displays a plurality of cursors simultaneously within 
said display region and controls a position of each ss 
of the cursors independently. 

14. The presentation system as defined in any one of 



claims 6 to 13, wherein said cursor control means 
controls a cursor position to indicate the detected 
pointing position, at a position having no overlap 
with said image region of said pointing image. 

15. The presentation system as defined in any one of 
claims 6 to 14, wherein said cursor control means 
controls a cursor size to be displayed according to 
a tip size of an image region of said pointing image 
having a rod like shape. 

16. The presentation system as defined in any one of 
claims 6 to 15, wherein said cursor control means 
comprises position-fixing means for fixedly display- 
ing said cursor at a desired position within said dis- 
play region. 

17. The presentation system as defined in any one of 
claims 6 to 16, wherein said image display means 
comprises an image projection means for projecting 
an image on a screen, and is disposed in such a 
manner as to project an image from in front of or 
behind said screen. 

18. The presentation system as defined in any one of 
claims 6 to 17, wherein said image-acquisition 
means freely adjusts an image-acquisition area 
with respect to said display region. 

19. The presentation system as defined in any one of 
claims 6 to 18, further comprising illumination 
means for illuminating a low-brightness level region 
of said display region. 

20. The presentation system as defined in claim 17, 
comprising a polarizing plate which is provided in a 
light path of said image-acquisition means and 
which absorbs light projected from an image projec- 
tion means. 

21. The presentation system as defined in any one of 
claims 6 to 20, further comprising: 

function input means to which a predetermined 
function is allocated; and 
means for performing data processing accord- 
ing to a function being input. 

22. A computer useable information embodied on com- 
puter-readable medium comprising: 

information for detecting a pointing position of 
an image region of a pointing image included 
in a display region, based on an image-acqui- 
sition signal of said display region; and 
information for controlling a position of a cursor 
included within an image displayed by an image 
display means, based on the detected pointing 
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position. 

23. The computer useable information embodied on 
computer-readable medium as defined in claim 22, 
wherein said information for controlling a cursor 
comprises information for controlling a cursor posi- 
tion to indicate the detected pointing position, at a 
position having no overlap with said image region 
of said pointing image. 

24. The computer useable information embodied on 
computer-readable medium as defined in claim 22 
or 23, wherein said information for controlling a cur- 
sor comprises information for controlling a cursor 
size to be displayed according to a tip size of said 
image region of said pointing image having a rod 
like shape. 

25. The computer useable information embodied on 
computer-readable medium as defined in any one 
of claims 22 to 24, wherein said information for con- 
trolling a cursor comprises position-fixing informa- 
tion for fixedly displaying said cursor at a desired 
position within said display region. 

26. A computer useable information embodied on com- 
puter-readable medium for controlling a presenta- 
tion system comprising: 

reference-value setting information for setting 
a reference value for separating an image re- 
gion of a pointing image included within a dis- 
play region from an image-acquisition signal of 
image-acquisition means; 
position detection information for detecting a 
pointing position from said image region of said 
pointing image, by comparing said image-ac- 
quisition signal and said reference value and 
thus extracting said image region of said point- 
ing image; and 

cursor control information for controlling a po- 
sition of a cursor included within an image dis- 
played by image display means, based on the 
detected pointing position, 
wherein said reference-value setting informa- 
tion comprises information for modifying said 
reference value. 

27. The computer useable information embodied on 
computer-readable medium as defined in claim 26, 
further comprising calibration pattern control infor- 
mation for displaying a predetermined calibration 
pattern image from said image display means to 
said display region, 

wherein said reference-value setting informa- 
tion comprises information for automatically setting 
a reference value for extracting an image region of 
said pointing image, based on data of said calibra- 



tion pattern image and said image-acquisition sig- 
nal. 

28. The computer useable information embodied on 
5 computer-readable medium as defined in claim 27, 

wherein said reference-value setting information 
comprises information for dividing said display re- 
gion intoa plurality of regions and automatically set- 
ting an individual reference value for each of the di- 
10 vided regions. 

29. A pointing position detection method comprising: 

an image-acquisition step of acquiring a picture 
'5 of a display region in which an image is dis- 

played; and 

a position detection step of detecting a pointing 
position from an image region of a pointing im- 
age included within said display region. 

20 

30. A cursor position control method comprising: 

an image display step of displaying an image; 
an image-acquisition step of acquiring an im- 
25 age of a display region in which an image is dis- 

played; 

a position detection step of detecting a pointing 
position of an image region of a pointing image 
included within said display region from an im- 
30 age-acquisition signal of said image-acquisi- 

tion means; and 

a cursor control step of controlling a position of 
a cursor included within the image displayed by 
said image display means, based on the detect- 
35 ed pointing position. 

31. A pointing position detection method comprising: 

an image-acquisition step of acquiring an im- 
40 age of a display region in which an image is dis- 

played; and 

a position detection step of comparing an im- 
age-acquisition signal with a reference value 
for separating an image region of a pointing im- 
45 age and thus extracting said image region of 

said pointing image, then detecting a pointing 
position from said image region of said pointing 
image, 

wherein said position detection step comprises 
50 a step of modifying said reference value for 

separating said image region of said pointing 
image included within said display region from 
said image-acquisition signal of an image-ac- 
quisition means. 

55 

32. A method of detecting a pointing position for a pres- 
entation, comprising: 
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an image-acquisition step of acquiring an im- 
age of a display region in which an image is dis- 
played; and 

a position detection step of comparing an im- 
age-acquisition signal with a reference value s 
for separating an image region of a pointing im- 
age and thus extracting said image region of 
said pointing image, then detecting a pointing 
position from said image region of said pointing 
image; and 10 
a cursor control step of controlling a position of 
a cursor included within an image displayed by 
an image display means, based on said detect- 
ed pointing position, 

wherein said position detection step comprises '£ 
a step of modifying said reference value for 
separating said image region of said pointing 
image included within said display region from 
said image-acquisition signal of an image-ac- 
quisition means. 20 

33. A method of detecting a pointing position for a pres- 
entation, comprising: 

an image-acquisition step of acquiring an im- 25 
age of a display region in which an image is dis- 
played; and 

a step of detecting a pointing position from an 
image region of a pointing image, based on an 
image-acquisition signal. so 
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